basic idea is to use Tushare to obtain the stock code, in order to obtain a link to the company's information, using requests to obtain the Web page source code, using BS4 to parse the structure of the Web page, find the necessary information, and then print or save.Summed up, here with the TUSHARE,REQUESTS,BS4, and other basic content. The demo code prints the information directly in the window, or you can try to save the information in a file such as Excel,csv.The next step is the code sectio
Crawl the ' Best University Network ', extracting the names and scores of the top 20 universities in 20171 #Coding:utf-82 ImportRequests3 fromBs4ImportBeautifulSoup4 ImportBS45 6 defgethtmltext (URL):7 Try:8r = Requests.get (URL, timeout=30)9 r.raise_for_status ()TenR.encoding =r.apparent_encoding One returnR.text A except: - return "fail" - the deffillunivlist (ulist, HTML): -Soup = BeautifulSoup (HTML,"Html.parser") -
(Str.indexof ("over")!=-1) Str=str.substr (4);
else str= "";
return sta+str;
}
function Startajaxtabs () {
for (Var i=0;i{
var Ulobj=document.getelementbyid (Arguments[i]);
Ulist=ulobj.getelementsbytagname ("Li");
for (Var j=0;j{
var thelist=ulist[j];
if (thelist.parentnode.parentnode!=ulobj) continue;//only the first layer of Li effective fixed 2006.9.29
var ulistlink=thelist.getelementsbytagna
it is also wrong to place this statement in the construction method. Public
String Login () {
req = Servletactioncontext.getrequest ();//req's acquisition must implement
user = new User () in a specific method;
User.setuid (UID);
User.setpassword (password);
if (userdao.islogin (user)) {
req.getsession (). setattribute ("user", user);
return SUCCESS;
}
return LOGIN;
}
Public
); user.setpassword (password); if (userdao.islogin (user)) { req.getsessiOn (). SetAttribute ("user", user); returnSUCCESS; } return LOGIN; } publicstringqueryall () { req=servletactioncontext.getrequest (); The acquisition of //req must be achieved in a specific way ulist=userdao.queryall (); req.getsession (). SetAttribute ("UList", ulist); returnSUCCESS; } /
; Private HttpServletRequest Requset = Servletactioncontext.getrequest (); It is wrong to place this statement in this position, and it is also wrong to put this statement in the constructor. Public String Login () { requset = Servletactioncontext.getrequest ();//Requset The acquisition must be implemented in a specific way user = new User (); User.setuid (UID); User.setpassword (password); if (userdao.islogin (user)) { requset.getsession
#采用request-BS4 Route realizes Chinese University ranked Directional crawler # optimization of mixed-output problems in China and English import requests from BS4 import beautifulsoup import BS4 #import BS4 to use its Label type definition
def gethtmltext (URL): try:r = requests.get (URL, timeout =) # print (R.status_code) 200 is normal, all other information is incorrect R.raise_for_status () r.encoding = r.apparent_encoding return r.text except:return ' Get Faile D. '
def fillunivlist (
achieved in a specific way user=newuser (); user.setuid (UID); user.setpassword (password); if (userdao.islogin (user)) { req.getsessiOn (). setattribute ("user", user); returnSUCCESS; } return LOGIN; NBSP;NBSP;NBSP;NBSP} publicstringqueryall () { req=servletactioncontext.getrequest () the acquisition of //req must be achieved in a specific method ulist=userdao.queryall (); req.getsession (). setattribute ("Ulis
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.